Systems and methods for anatomic structure segmentation in image analysis

ABSTRACT

Systems and methods are disclosed for anatomic structure segmentation in image analysis, using a computer system. One method includes: receiving an annotation and a plurality of keypoints for an anatomic structure in one or more images; computing distances from the plurality of keypoints to a boundary of the anatomic structure; training a model, using data in the one or more images and the computed distances, for predicting a boundary in the anatomic structure in an image of a patient&#39;s anatomy; receiving the image of the patient&#39;s anatomy including the anatomic structure; estimating a segmentation boundary in the anatomic structure in the image of the patient&#39;s anatomy; and predicting, using the trained model, a boundary location in the anatomic structure in the image of the patient&#39;s anatomy by generating a regression of distances from keypoints in the anatomic structure in the image of the patient&#39;s anatomy to the estimated boundary.

RELATED APPLICATION

This application claims priority to U.S. Provisional Application No.62/503,838, filed on May 9, 2017, the entire disclosure of which ishereby incorporated by reference in its entirety.

TECHNICAL FIELD

Various embodiments of the present disclosure relate generally tomedical imaging and related methods. Specifically, particularembodiments of the present disclosure relate to systems and methods foranatomic structure segmentation in image analysis.

BACKGROUND

The problem of partitioning an image into multiple segments commonlyoccurs in computer vision and medical image analysis. A currently usedapproach is to automate this process using a convolutional neuralnetwork (CNN), which is trained to predict a class label for each imageelement (e.g., pixel or voxel). CNNs typically include multipleconvolutional layers, which pass the input (e.g., an image or a portionof an image) through a set of learnable filters and nonlinear activationfunctions. The use of convolutional operations makes CNNs equivariant totranslations. For example, translated versions of the input may lead toproportionally translated versions of the predicted segmentation labels.The set of layers with convolutions of different strides may enable CNNsto express long-range interactions in the image in terms of local, shortrange statistics.

The segmentation boundary of current CNNs, however, may be accurate upto the level of an image element (e.g., a pixel or a voxel). In manyimaging applications, a quantization error may be introduced by placingthe segmentation boundary at pixel or voxel locations. In some cases, itmay be known (e.g., as a priori) that a structure of interest does notcontain holes and may exist as one connected component. However, theseassumptions may not be integrated into the CNN such that the predictedlabels may have spurious components and holes in the segmented objects.Thus, there is a desire to build models such as CNNs that can achievesub-pixel or sub-voxel accurate segmentations and can predict labels forsingle connected components without holes or disconnected structures.

The present disclosure is directed to overcoming one or more of theabove-mentioned problems or interests.

SUMMARY

According to certain aspects of the present disclosure, systems andmethods are disclosed for anatomic structure segmentation in imageanalysis. One method of anatomic structure segmentation in imageanalysis includes: receiving an annotation and a plurality of keypointsfor an anatomic structure in one or more images; computing distancesfrom the plurality of keypoints to a boundary of the anatomic structure;training a model, using data in the one or more images and the computeddistances, for predicting a boundary in the anatomic structure in animage of a patient's anatomy; receiving the image of the patient'sanatomy including the anatomic structure; estimating a segmentationboundary in the anatomic structure in the image of the patient'sanatomy; and predicting, using the trained model, a boundary location inthe anatomic structure in the image of the patient's anatomy bygenerating a regression of distances from keypoints in the anatomicstructure in the image of the patient's anatomy to the estimatedboundary.

According to another embodiment, a system is disclosed for anatomicstructure segmentation in image analysis. The system includes a datastorage device storing instructions for anatomic structure segmentationin image analysis; and a processor configured to execute theinstructions to perform a method including the steps of: receiving anannotation and a plurality of keypoints for an anatomic structure in oneor more images; computing distances from the plurality of keypoints to aboundary of the anatomic structure; training a model, using data in theone or more images and the computed distances, for predicting a boundaryin the anatomic structure in an image of a patient's anatomy; receivingthe image of the patient's anatomy including the anatomic structure;estimating a segmentation boundary in the anatomic structure in theimage of the patient's anatomy; and predicting, using the trained model,a boundary location in the anatomic structure in the image of thepatient's anatomy by generating a regression of distances from keypointsin the anatomic structure in the image of the patient's anatomy to theestimated boundary.

In accordance with yet another embodiment, a non-transitory computerreadable medium for use on a computer system containingcomputer-executable programming instructions for performing a method ofanatomic structure segmentation in image analysis is provided. Themethod includes: receiving an annotation and a plurality of keypointsfor an anatomic structure in one or more images; computing distancesfrom the plurality of keypoints to a boundary of the anatomic structure;training a model, using data in the one or more images and the computeddistances, for predicting a boundary in the anatomic structure in animage of a patient's anatomy; receiving the image of the patient'sanatomy including the anatomic structure; estimating a segmentationboundary in the anatomic structure in the image of the patient'sanatomy; and predicting, using the trained model, a boundary location inthe anatomic structure in the image of the patient's anatomy bygenerating a regression of distances from keypoints in the anatomicstructure in the image of the patient's anatomy to the estimatedboundary.

Additional objects and advantages of the disclosed embodiments will beset forth in part in the description that follows, and in part will beapparent from the description, or may be learned by practice of thedisclosed embodiments. The objects and advantages of the disclosedembodiments will be realized and attained by means of the elements andcombinations particularly pointed out in the appended claims.

It is to be understood that both the foregoing general description andthe following detailed description are exemplary and explanatory onlyand are not restrictive of the disclosed embodiments, as claimed.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are incorporated in and constitute apart of this specification, illustrate various exemplary embodiments andtogether with the description, serve to explain the principles of thedisclosed embodiments.

FIG. 1 is a block diagram of an exemplary system and network foranatomic structure segmentation in image analysis, according to anexemplary embodiment of the present disclosure.

FIGS. 2A and 2B are flowcharts of an exemplary method for anatomicstructure segmentation in image analysis, according to an exemplaryembodiment of the present disclosure.

FIGS. 3A and 3B are flowcharts of an exemplary embodiment of the methodof FIGS. 2A and 2B, as applied to segmentation of coronary arteries,according to an exemplary embodiment of the present disclosure.

DESCRIPTION OF THE EMBODIMENTS

Reference will now be made in detail to the exemplary embodiments of thepresent disclosure, examples of which are illustrated in theaccompanying drawings. Wherever possible, the same reference numberswill be used throughout the drawings to refer to the same or like parts.

As described above, the accuracy of the segmentation boundariesdetermined by current approaches may be limited to an image element,e.g., a pixel or a voxel. In these cases, errors may be introduced byplacing the segmentation boundary at voxel locations. In some cases,current prediction models may not take into account some assumptionssuch as that structure of interest does not contain holes ordisconnected structures. Thus, there is a desire to build models thatcan predict segmentation boundaries with sub-pixel or sub-voxel accuracyand/or ensure important assumptions to be integrated in the model.

The present disclosure is directed to providing accurate prediction ofsegmentation boundary locations. In one embodiment, the presentdisclosure may include both a training phase and a testing (or usage)phase to estimate a segmentation boundary. One or more parameters of alearning system for developing a trained model may be optimized duringthe training phase. During the testing phase, an unseen or seen imagecan be segmented with the trained model.

For example, the disclosed systems and methods may be applied tosegmenting anatomy in received image(s) of a patient of interest anddetermining the boundary of a structure of interests at a sub-pixel orsub-voxel level. As used herein, a boundary of a structure may include aboundary of a segment of the structure. In one embodiment, the trainingphase may include developing a model for predicting a distance from akeypoint in a structure of interest to a boundary of the structure ofinterest or a segment thereof. For example, the training phase mayinvolve receiving a plurality of keypoints with known locations in thestructure of interest and computing the distances from the keypoints toa boundary of the structure of interest of a segment thereof (e.g.,based on the known locations). Then a model (e.g., a CNN model) may betrained based on the locations of the keypoints, the computed distances,and/or the data in the received images. The trained model may regress asample distance or predict an indirect representation of the sampledistance. The regression from the trained model may be a continuousvalue, thus allowing for predicting boundary locations based on theregressed distance with sub-pixel or sub-voxel accuracy.

In one embodiment, a testing phase may include receiving images of apatient's anatomy. The patient may be a patient of interest, e.g., apatient desiring a diagnostic test. The testing phase may involveestimating a boundary of a structure of interest based on one or moreimages of the patient's anatomy and predict the boundary locations byregressing the distances from keypoints in the structure of interest tothe estimated boundary using the model developed from the trainingphase.

As used herein, the term “exemplary” is used in the sense of “example,”rather than “ideal.” Although this exemplary embodiment is written inthe context of medical image analysis, the present disclosure mayequally apply to any non-medical image analysis or computer visionevaluation.

Referring now to the figures, FIG. 1 depicts a block diagram of anexemplary environment of a system and network for anatomic structuresegmentation in image analysis. Specifically, FIG. 1 depicts a pluralityof physicians 102 and third party providers 104, any of whom may beconnected to an electronic network 100, such as the Internet, throughone or more computers, servers, and/or handheld mobile devices.Physicians 102 and/or third party providers 104 may create or otherwiseobtain images of one or more patients' cardiac, vascular, and/or organsystems. The physicians 102 and/or third party providers 104 may alsoobtain any combination of patient-specific information, such as age,medical history, blood pressure, blood viscosity, etc. Physicians 102and/or third party providers 104 may transmit the cardiac/vascular/organimages and/or patient-specific information to server systems 106 overthe electronic network 100. Server systems 106 may include storagedevices for storing images and data received from physicians 102 and/orthird party providers 104. Server systems 106 may also includeprocessing devices for processing images and data stored in the storagedevices. Alternatively or in addition, the anatomic structuresegmentation of the present disclosure (or portions of the system andmethods of the present disclosure) may be performed on a localprocessing device (e.g., a laptop), absent an external server ornetwork.

FIGS. 2A and 2B describe exemplary methods for performing anatomicstructure segmentation using a learning system. FIGS. 3A and 3B aredirected to specific embodiments or applications of the methodsdiscussed in FIGS. 2A and 2B. For example, FIG. 3A and FIG. 3B describean embodiment for the segmentation of blood vessels using a learningsystem. All of the methods may be performed by server systems 106, basedon information, images, and data received from physicians 102 and/orthird party providers 104 over electronic network 100.

FIGS. 2A and 2B describe exemplary methods for anatomic structuresegmentation in image analysis. In one embodiment, the anatomicstructure segmentation may include two phases: a training phase and atesting phase. The training phase may involve training a learning system(e.g., a deep learning system) to predict a boundary location of astructure of interest or a segment thereof with sub-pixel or sub-voxelaccuracy. The testing phase may include predicting a boundary locationof the structure of interest or a segment thereof in a newly receivedimage.

FIG. 2A is a flowchart of an exemplary training phase method 200 fortraining a learning system (e.g., a deep learning system) to predict aboundary location, according to various embodiments. Method 200 mayprovide the basis for a testing phase in method 210 of FIG. 2B, foranatomic structure segmentation of an imaged structure of interest of aspecific patient. Method 200 may include one or more of steps 201-207shown in FIG. 2A. In some embodiments, method 200 may include repeatingone or more of steps 201-207, e.g., repeating steps 201 through 207, forone or more times.

In some embodiments, step 201 of method 200 may include receiving one ormore images and/or image data in an electronic storage medium (e.g.,hard drive, network drive, cloud drive, mobile phone, tablet, database,etc.). In a medical context, these images may, for instance, be from amedical imaging device, e.g., computed tomography (CT), positronemission tomography (PET), single-photon emission computerizedtomography (SPECT), magnetic resonance imaging (MRI), microscope,ultrasound, (multi-view) angiography, etc. In one embodiment, multipleimages for a single patient may be used. In a further embodiment, theimages may comprise a structure of a patient's anatomy. In otherembodiments, the images may be of numerous individuals having similaranatomical features or numerous individuals having different anatomicalfeatures. In a non-medical context, these images may be from any source,e.g., a camera, satellite, radar, lidar, sonar, telescope, microscope,etc. In the following disclosure, images received in step 201 may bereferred to as “training images.”

In some embodiments, step 202 may include receiving an annotation forone or more structures of interest in one or more of the trainingimages. In some cases, one or more of the training images may include astructure of interest, e.g., an anatomic structure of the patient. Inone example, all of the training images may include the structure ofinterest. In one embodiment, all training images may be annotated. Thistype of embodiment may be referred to as “supervised learning.” Anotherembodiment may include only a subset of the training images withannotations. This type of scenario may be referred to as“semi-supervised learning.” In one embodiment, the structures ofinterest may include a blood vessel or tissue of the patient. In such acase, annotation(s) may include labels for vessel names (e.g., rightcoronary artery (RCA), left anterior descending artery (LAD), leftcircumflex artery (LCX), etc.), vessel landmarks (e.g., aortic valvepoint, ostia points, bifurcations, etc.), estimated vessel location,flags (e.g., noted portions where imaging is ambiguous or boundaries orunclear), etc. In some embodiments, the annotation(s) may be in variousformats, including, but not limited to, meshes, voxels, implicit surfacerepresentations, or point clouds.

In some embodiments, step 203 may include receiving a plurality ofkeypoints in the structure of interest. The locations of the keypointsin the structure of interest may be known (e.g., based on the annotationfor the structure of interest and/or data in the received images). Forexample, one or more of the keypoints may be known to be inside thestructure of interest, on the boundary of the structure of interest, orclose to the boundary of the structure of interest. In some embodiments,the boundary location (e.g., rough boundary location) of the structureof interest and/or the locations of the keypoints may be determined,and/or the keypoints with known locations in the structure of interestmay be retrieved, by fitting a shape model to the structure of interest.

In some embodiments, step 204 may include defining a mapping from theimage coordinates in the training images to a Euclidean space. The imagecoordinates may be continuous. The image coordinates may be theintersection points of the 3D structure of interest and/or the rays. Theimage coordinates may be in rays in the Euclidean space. There may beone of the key points received in step 203 on each of the rays.

In one embodiment, a given keypoint may be known to be inside thestructure of interest. In such as a scenario, step 204 may includeformulating a mapping in polar form with the given keypoint at thecenter. In this setting, the image coordinates along equally long raysin equiangular directions that originate from the keypoint may bechosen.

As another scenario, the mapping in step 204 may include a mapping frommultiple polar dimensions to a Euclidean space. For example, it may bepossible to sample from a spherical coordinate system to parameterize astructure of interest (e.g., a 3-dimensional (3D) structure ofinterest). In this embodiment, two cyclic dimensions may map to threeEuclidean dimensions given by the distance along the ray and the twodimensions that are associated with the two cyclic dimensions.

As another scenario, the mapping in step 204 may include a mapping frompolar form in one dimension and linear form in one or more additionaldimensions to a Euclidean space. For example, a tube like structure maybe represented by landmark points on a sequence of closed curves. Theclosed curves may then be represented in polar form, whereas thedirection along the tube may be linear. Thus, this mapping may amount toa three dimensional Euclidean space, where the first dimensionrepresents the distance along the sampled ray, the second dimensionrepresents the cyclic dimension, and the third dimension represents thelinear dimension along the tube.

In one embodiment, a set of keypoints close to the image surface may bereceived in step 203. For example, the set of keypoints may be arrangedalong a set of closed curves on a given 3D mesh or a 3D implicitrepresentation. Then, step 204 may include defining a set of equallylong rays that each includes a keypoint and is directed perpendicular tothe 3D mesh, and refining the boundary of the structure of interest may.Then when mapping to a Euclidean space in step 204, the distance alongthe defined ray may represent one of the Euclidean dimensions, whereasthe keypoints on the closed curve may represent the second dimension.

In some embodiments, step 205 may include, for each of the ray mapped instep 204, computing a distance from the keypoint on the ray to theboundary of the structure of interest. The computed distances may be thetarget values of the learning system.

In some embodiments, step 206 may include, for each target value,determining image intensities along the rays defined in step 204. Thedefined rays are associated with keypoints on the rays and are thusassociated with the target values. Step 206 may further include ensuringthat the rays associated with target values are at fixed locations. Forexample, a first ray associated with a first target value may be at afirst location that is fixed. The other rays associated with othertarget values may have a coordinate relative to the first ray, thus thelocations of these other rays may be fixed as well based the location ofthe first ray. In one embodiment, the input for each of the target valuedistances may be cyclic transformations of each other.

In some embodiments, step 207 may include training a model (e.g., a CNNmodel) for predicting the segmentation boundary location of thestructure of interest in a newly received image. For example, the modelmay be trained to predict the distance from a keypoint to thesegmentation boundary of the structure of interest. In one embodiment,the model may be trained for regressing the distance. The regressionvalue may be continuous so that the boundary location may be predictedwith sub-pixel or sub-voxel accuracy. In certain embodiments, the modelmay be trained for predicting an indirect representation of thedistance. For example, the model may quantize the ray into a pluralityof small bins and/or may predict the bin that represents the distance.

FIG. 2B is a block diagram of an exemplary testing phase (or usagephase) method 210 for predicting a boundary of a segmentation of astructure of interest in a specific patient image, according to anexemplary embodiment of the present disclosure. Method 210 may includeone or more of steps 211-215. In one embodiment, the boundary of thesegmentation may be predicted using a trained model (e.g., from method200).

In some embodiments, step 211 may include receiving one or more imagesfrom a patient in an electronic storage medium (e.g., hard drive,network drive, cloud drive, mobile phone, tablet, database, etc.). Inone embodiment, the images may include medical images, e.g., images maybe from any medical imaging device, e.g., CT, MR, SPECT, PET,microscope, ultrasound, (multi-view) angiography, etc. In oneembodiment, training images (e.g., of method 200) may include imagesacquired from one patient, and step 211 may include receiving imagesalso of that one patient. Alternately or additionally, step 211 mayinclude receiving one or more images from a non-medical imaging device,e.g., a camera, satellite, radar, lidar, sonar, telescope, microscope,etc. In the following steps, images received during step 211 may bereferred to as “testing images.”

In some embodiments, step 212 may include estimating a segmentationboundary of the structure of interest, or the location or boundary ofanother object in the testing images (e.g., an object different from thestructure of interest). The estimated boundary or the location orboundary of another object may be used to initialize an automatedsegmentation system (e.g., centerlines). A set of keypoints may beretrieved from this initial segment. In one embodiment, the set ofkeypoints may include a keypoint inside the structure of interest.

In some embodiments, step 213 may include defining a mapping from theimage coordinates in the testing images to a Euclidean space. The imagecoordinates may be continuous. The image coordinates may be in rays inthe Euclidean space. There may be a keypoint (e.g., one of the keypointsretrieved in step 202) on each of the rays may include a keypoint. Inone embodiment, the input of this mapping may be analogous to the inputin the mapping in step 204.

In some embodiments, step 214 may include predicting, using the modeltrained by method 200, the boundary in the structure of interest. In oneembodiment, the predicting may include regressing the distance from thekeypoint on the rays defined in step 213 to the estimated boundary. Insome cases, the regression may be a continuous value, and thus theboundary may be predicted with sub-pixel or sub-voxel accuracy. In oneembodiment, step 214 may further include obtaining a surface from thepredicted boundary (e.g., boundary point cloud). The surface may beobtained using a surface reconstruction method, such as Poisson surfacereconstruction.

In some embodiments, step 215 may include outputting the predictedboundary (e.g., the complete segmentation boundary) of the structure ofinterest to an electronic storage medium (e.g., hard drive, networkdrive, cloud drive, mobile phone, tablet, database, etc.). Step 215 mayfurther include displaying the output result on an interface.

FIGS. 3A and 3B are directed to specific embodiments or applications ofthe exemplary methods discussed in FIGS. 2A and 2B. For example, FIG. 3Aand FIG. 3B, respectively, describe an exemplary training phase andtesting phase for segmentation of coronary arteries in image analysis.

The accuracy of patient-specific segmentation of blood vessels, e.g.,coronary arteries, may affect medical assessments such as blood flowsimulation or calculation of geometric characteristics of blood vessels.If the accuracy of the segmentation is not sufficient, e.g., limited tothe level of an image element (e.g., a pixel or a voxel), the medicalassessments may generate spurious results. FIGS. 3A and 3B showexemplary methods for segmentation of coronary arteries with sub-pixelor sub-voxel accuracy. Although coronary arteries are used in theexemplary methods herein, the methods illustrated in FIGS. 3A and 3B mayalso be used for segmentation of other types of blood vessels oranatomic structures other than blood vessels.

FIG. 3A is a flowchart of an exemplary method 300 for a training phasedesigned to provide the basis for sub-voxel segmentation of coronaryarteries, according to various embodiments. Method 300 may include oneor more of steps 301-308 shown in FIG. 3A. In some embodiments, method200 may include repeating one or more of steps 301-308, e.g., repeatingsteps 301 through 308, for one or more times.

In some embodiments, step 301 may include receiving one or more imagesof coronary arteries in an electronic storage medium (e.g., hard drive,network drive, cloud drive, mobile phone, tablet, database, etc.). Theseimages may be from a medical imaging device, such as CT, MR, SPECT, PET,ultrasound, (multi-view) angiography, etc. These images may be referredto as “training images.”

In some embodiments, step 302 may include receiving annotations for thecoronary arteries in one or more of the training images. For examples,the annotations may include vessel lumen boundary and/or the vessellumen centerline(s). In one embodiment, step 303 may include receivingor generating a geometric mesh of the coronary vessels represented inthe received images. The geometric mesh may be specified as a set ofvertices and edges. Alternately or additionally, step 303 may includereceiving a centerline of the coronary vessels. The centerline may alsobe represented as a set of vertices that may be connected by edges.

In some embodiments, step 303 may include transforming the trainingimage data (e.g., the geometric mesh, vertices, edges, centerline, etc.)into a curvilinear planar representation (CPR). The transformation mayallow simplification of the blood vessel segmentation process. Forexample, a set of planes (e.g., frames) may be extracted along thecenterline (e.g., orthogonal to the centerline) to constitute a 3Dvolume. In one embodiment, the 3D volume may comprise a CPR, with acoordinate system frame of reference defining two dimensions and thecenterline length defining a third dimension. In one embodiment, thecurvilinear planar representation may eliminate degrees of freedom(e.g., the curvature of the centerline), which may not be relevant forpredicting one or more parameters of the coronary vessels. For example,the curvature of the centerline may be irrelevant for determining aparameter related to the location of the coronary vessels' lumenboundary.

In some embodiments, step 304 may include defining keypoints based onthe image data. For example, step 304 may include defining points on thecenterline of the coronary arteries as keypoints. Such defined keypointsmay be assumed to be inside the blood vessel. These keypoints may notnecessarily be centered. In some cases, nevertheless, these keypointsmay, by construction, be in the center of each frame.

In some embodiments, step 305 may include, for each frame, defining amapping from the image coordinates in the testing images to a Euclideanspace. In one embodiment, the mapping may be defined using polarsampling within the frame. In certain embodiments, defining the mappingmay include determining CPR intensity values in a set of angulardirections around a keypoint defined in step 304. The determined CPRintensity value may be arranged, e.g., so that the radial and angularcoordinates map to a 2-dimensional (2D) image. For example, a discreteset of samples specified by a radial and an angular component of theframe may be mapped to the row and the column of a 2D image that indexesthe radial and the angular component. Each row of the CPR intensityvalues may be defined as a radial coordinate and each column of the CPRintensity values may be defined as an angular coordinate.

In some embodiments, step 306 may include defining target regressionvalues. The target regression values may be defined as the distancesfrom a given keypoint to the boundary of the blood vessel lumen in eachangular direction. In one embodiment, step 306 may include definingtarget regression values in r angular directions. For example, for agiven target distance value in an r angular direction, the 2D imagecreated in step 305 may be cyclically rotated so that the columnassociated with the r angular direction that is associated with thegiven target regression value is the first column. For example, whenpredicting all r target values, of which each is associated with adifferent column of an input image, the columns may be cyclicallyrotated. If the image is rotated by r columns, the r-th column becomesthe first column and the same model that is used to predict the targetvalue for the first column may be applied for the target value of r-thcolumn that is in the first column after the cyclic rotation.

In some embodiments, step 307 may include training a model (e.g., a CNNmodel) for predicting the distance from a given keypoint to the boundaryof the blood vessel lumen. In one embodiment, the trained model maypredict the mapping from each of the 2D images created in step 305 tothe associated target distance value. The loss function may be specifiedto minimize the mean squared error between the predicted and the targetdistance. As used herein, a loss function may specify the error betweenthe prediction and the target value, and is an integral part of anobjective function that is optimized to learn the suitable modelparameters. For example, a loss function may be a mean squared error,e.g., the mean of the squares of the difference between the predictionand the target value.

FIG. 3B is a block diagram of an exemplary method 310 for a testingphase that may provide a sub-voxel segmentation of a patient's bloodvessels (e.g., coronary arteries), according to one embodiment. In someembodiments, step 311 may include receiving image data of a patient'scoronary artery in an electronic storage medium (e.g., hard drive,network drive, cloud drive, mobile phone, tablet, database, etc.).

In some embodiments, step 312 may include receiving a prediction of thecenterline of the blood vessels using, for example, a centerlinedetection algorithm. In one embodiment, step 312 may includetransforming the received image or image data into a CPR. Thetransformation may allow simplification of the blood vessel segmentationprocess. For example, a set of planes (e.g., frames) may be extractedalong the centerline (e.g., orthogonal to the centerline) of the bloodvessel lumen to constitute a 3D volume (e.g., CPR). In one embodiment,the 3D volume may comprise a CPR, with a coordinate system frame ofreference defining two dimensions and the centerline length defining athird dimension. The transformation parameters (e.g., translation,scale, rotation) may be stored.

In some embodiments, step 313 may include defining points on thecenterline of the blood vessels as keypoints.

In some embodiments, step 314 may include, for each of the frame definedin step 312, defining a mapping of the image coordinates in thepatient's image to a Euclidean space. For example, the mapping may bedefined using polar sampling within the frame. This step may beanalogous to one or more steps in method 300.

In some embodiments, step 315 may include determining CPR intensityvalues in a set of angular directions around a keypoint defined in step313. The determined CPR intensity value may be arranged so that theradial and angular coordinates map to a 2-dimensional (2D) image. The 2Dimage may be cyclically rotated so that the column associated with the rangular direction that is associated with the given target distancevalue is the first column. Step 315 may further include creatingcyclically rotated (in r angular directions) versions of the 2D image.

In some embodiments, step 316 may include predicting the segmentationboundary of the patient's coronary arteries using the model trained inmethod 300. In one embodiment, step 316 may include predicting thedistance associated with the first column in each of the rotated imagescreated in step 315, thus predicting landmark points of the boundary inCPR representation. In one embodiment, step 316 may include generatingan anatomic model of the patient's imaged coronary artery. The anatomicmodel may include a final lumen segmentation with a sub-pixel orsub-voxel accuracy. For example, step 317 may include transforming thepredicted landmark point(s) from the CPR representation to the original3D image space. The orientation and position of each frame along thecenterline may be determined from the creation of a CPR. For example,the orientation and position may be determined and stored in step 312.In one embodiment, the 3D points may be computed from the CPR, and any3D surface reconstruction method (e.g., Poisson surface reconstruction)may be applied to the point cloud of the landmark point(s) to constructthe anatomic model or final lumen segmentation of the patient's coronaryarteries.

In some embodiments, step 317 may include outputting the anatomic modeland/or the complete segmentation boundary of the blood vessels to anelectronic storage medium (e.g., hard drive, network drive, cloud drive,mobile phone, tablet, database, etc.) and/or display.

Other embodiments of the invention will be apparent to those skilled inthe art from consideration of the specification and practice of theinvention disclosed herein. It is intended that the specification andexamples be considered as exemplary only, with a true scope and spiritof the invention being indicated by the following claims.

1-20. (canceled)
 21. A computer-implemented method of anatomic structuresegmentation in image analysis, the method comprising: receiving firstimage data of an anatomic structure of a patient, the first image dataincluding one or more frames along a centerline formed by a plurality ofpixels or voxels; obtaining an estimation of a boundary of the anatomicstructure; transforming each of the one or more frames into acurvilinear planar representation (CPR) based on a geometry associatedwith the estimation; determining one or more keypoints in each framebased on the estimation of the boundary; and generating, using a trainedmodel, a sub-pixel or sub-voxel boundary prediction of the anatomicstructure in the first image data based on the first image data, thedetermined one or more keypoints, and the estimation of the boundary ofthe anatomic structure, wherein the trained model has been trained,based on (i) second image data of the anatomic structure of one or moreindividuals, (ii) one or more estimations of the boundary of theanatomic structure of the one or more individuals, and (iii) distancesbetween the estimations of the boundary and a sub-pixel or sub-voxellocation of the boundary of the anatomic structure of the one or moreindividuals, to learn an association between the estimation of theboundary and the sub-pixel or sub-voxel location of the boundary. 22.The method of claim 21, wherein the association between the estimationof the boundary and the sub-pixel or sub-voxel location of the boundaryis based on a regression of intensity values of pixels or voxels in thesecond image data along each of a plurality of radial angular directionsout from one or more keypoints from the one or more estimations of theboundary.
 23. The method of claim 22, wherein the regression is acontinuous regression.
 24. The method of claim 22, the regression on theintensity values includes: performing the regression on a first columnof columns of pixels or voxels in a respective segment of each frame inthe second image data; cyclically rotating the columns of pixels orvoxels so that a last column is rearranged to be a replacement of thefirst column, and so that any remaining columns are shifted by one; andrepeating the performing of the regression and the cyclical rotationuntil the regression has been performed on each column of the frame. 25.The method of claim 21, wherein the transforming of each frame into CPRis performed in segments.
 26. The method of claim 21, wherein theestimation of the boundary of the anatomic structure is in a format of amesh, a voxel, an implicit surface representation, or a point cloud. 27.The method of claim 21, wherein a relative location of the one or morekeypoints is fixed relative to each frame.
 28. The method of claim 21,wherein: the sub-pixel or sub-voxel boundary prediction includes a pointcloud; and the method further comprises generating a continuous surfaceof the boundary based on the point cloud, the continuous surface havingsub-pixel or sub-voxel accuracy.
 29. The method of claim 21, wherein theanatomic structure comprises a blood vessel.
 30. The method of claim 29,wherein the boundary is associated with a vessel lumen boundary, avessel lumen surface, or a combination thereof.
 31. A system foranatomic structure segmentation in image analysis, comprising: at leastone processor; and a memory operatively connected to the processor, andstoring: a trained model that has been trained, based on (i) first imagedata of an anatomic structure of one or more individuals, (ii) one ormore estimations of a boundary of the anatomic structure of the one ormore individuals, and (iii) distances between the estimation of theboundary and a sub-pixel or sub-voxel location of the boundary of theanatomic structure of the one or more individuals, to learn anassociation between the estimation of the boundary and the sub-pixel orsub-voxel location of the boundary; and instructions executable by theprocessor to perform operations, including: receiving second image dataof an anatomic structure of a patient, the second image data includingone or more frames along a centerline formed by a plurality of pixels orvoxels; obtaining an estimation of a boundary of the anatomic structureof the patient; transforming each of the one or more frames into acurvilinear planar representation (CPR) based on a geometry associatedwith the estimation; determining one or more keypoints in each framebased on the estimation; and generating, using the trained model, asub-pixel or sub-voxel boundary prediction of the anatomic structure inthe second image data based on the second image data, the determined oneor more keypoints, and the estimation of the boundary of the anatomicstructure.
 32. The system of claim 31, wherein the association betweenthe estimation of the boundary and the sub-pixel or sub-voxel locationof the boundary is based on a regression of intensity values of pixelsor voxels in the first image data along each of a plurality of radialangular directions out from one or more keypoints from the one or moreestimations of the boundary.
 33. The system of claim 32, wherein theregression is a continuous regression.
 34. The system of claim 31,wherein: the sub-pixel or sub-voxel boundary prediction includes a pointcloud; and the operations further include generating a continuoussurface of the boundary based on the point cloud, the continuous surfacehaving sub-pixel or sub-voxel accuracy.
 35. A non-transitory computerreadable medium for use on a computer system containingcomputer-executable programming instructions for anatomic structuresegmentation in image analysis, the instructions executing operationscomprising: receiving first image data of an anatomic structure of apatient, the first image data including one or more frames along acenterline formed by a plurality of pixels or voxels; obtaining anestimation of a boundary of the anatomic structure; transforming each ofthe one or more frames into a curvilinear planar representation (CPR)based on a geometry associated with the estimation; determining one ormore keypoints in each frame based on the estimation of the boundary;and generating, using a trained model, a sub-pixel or sub-voxel boundaryprediction of the anatomic structure in the first image data based onthe first image data, the determined one or more keypoints, and theestimation of the boundary of the anatomic structure, wherein thetrained model has been trained, based on (i) second image data of theanatomic structure of one or more individuals, (ii) one or moreestimations of the boundary of the anatomic structure of the one or moreindividuals, and (iii) distances between the estimations of the boundaryand a sub-pixel or sub-voxel location of the boundary of the anatomicstructure of the one or more individuals, to learn an associationbetween the estimation of the boundary and the sub-pixel or sub-voxellocation of the boundary.
 36. The non-transitory computer readablemedium of claim 35, wherein the association between the estimation ofthe boundary and the sub-pixel or sub-voxel location of the boundary isbased on a regression of intensity values of pixels or voxels in thesecond image data along each of a plurality of radial angular directionsout from one or more keypoints from the one or more estimations of theboundary.
 37. The non-transitory computer readable medium of claim 35,wherein a relative location of the one or more keypoints is fixedrelative to each frame.
 38. The non-transitory computer readable mediumof claim 35, wherein: the sub-pixel or sub-voxel boundary predictionincludes a point cloud; and the method further comprises generating acontinuous surface of the boundary based on the point cloud, thecontinuous surface having sub-pixel or sub-voxel accuracy.
 39. Thenon-transitory computer readable medium of claim 35, wherein theanatomic structure comprises a blood vessel.
 40. The non-transitorycomputer readable medium of claim 39, wherein the boundary is associatedwith a vessel lumen boundary, a vessel lumen surface, or a combinationthereof.